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Abstract 



This technical report describes the development of fifth grade progress monitoring 
measures in the area of Passage Reading Fluency. This measure was designed to target the 
fluency component of a developmental model of reading. Twenty alternate forms were written 
by graduate students and reviewed by the lead author. The passages were piloted and mean 
scores were compared as a measure of difficulty. In response to these data, the passages were 
brought into closer alignment by identifying nine passages that showed similar difficulty level 
and adjusting the remaining passages to match this level. Data on the difficult of each passage 
and a summary of revisions are presented. 
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Introduction 

In this technical report, we describe the development alternate forms of fifth-grade 
Passage Reading Fluency measures as part of a comprehensive progress monitoring literacy 
assessment system developed in 2006 for use with students in Kindergarten through fifth grade. 
We begin with a brief overview of the two conceptual frameworks underlying the assessment 
system: progress monitoring and developmental theories of reading. We then provide context for 
how the Passage Reading Fluency measures fit into the full assessment system. Additional 
technical reports provide similar information about measures of Early Literacy such as Letter 
Names, Letter Sounds, and Phoneme Segmenting (Alonzo & Tindal, 2007) and Reading 
Comprehension (Alonzo, Liu, & Tindal, 2007). 

Conceptual Framework: Progress Monitoring and Literacy Assessment 

Early work related to curriculum-based measurement (CBM) led by Deno and Mirkin at 
the University of Minnesota ( c.f.a ., Deno & Mirkin, 1977) was instrumental in promoting the use 
of short, easily-administered assessments to provide educators with information about student 
skill development useful for instructional planning. In the three decades since, such progress 
monitoring probes as they have come to be called have increased in popularity, and they are now 
a regular part of many schools’ educational programs (Alonzo, Ketterlin-Geller, & Tindal, 2007). 
However, CBMs - even those widely used across the United States - often lack the psychometric 
properties expected of modern technically-adequate assessments. Although the precision of 
instrument development has advanced tremendously in the past 30 years with the advent of more 
sophisticated statistical techniques for analyzing tests on an item by item basis rather than relying 
exclusively on comparisons of means and standard deviations to evaluate comparability of 
alternate forms, the world of CBMs has not always kept pace with these statistical advances. 
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A key feature of assessments designed for progress monitoring is that alternate forms 
must be as equivalent as possible to allow meaningful interpretation of student performance data 
across time. Without such cross-form equivalence, changes in scores from one testing session to 
the next are difficult to attribute to changes in student skill or knowledge. Improvements in 
student scores may, in fact, be an artifact of the second form of the assessment being easier than 
the form that was administered first. The advent of more sophisticated data analysis techniques 
(such as the Rasch modeling used in this study) have made it possible to increase the precision 
with which we develop and evaluate the quality of assessment tools. In this technical report, we 
document the development of a progress monitoring assessment in reading, designed for use with 
students in Kindergarten through Grade 4. This assessment system was developed to be used by 
elementary school educators interested in monitoring the progress their students make in the area 
of early reading skill acquisition. 

Reading is a somewhat fluid construct, shifting over time from a focus on discrete skills 
necessary for working with language in both written and spoken forms, to those more complex 
combinations of skills associated with decoding, and finally to comprehension — a construct in 
which all prior literacy skills are called upon in the act of reading. Reading assessment typically 
follows this general progression as well (Reading First, 2006). Assessments of emerging literacy 
skills evaluate student mastery of the alphabetic principal. These tests measure students’ ability 
to correctly identify and/or produce letters and the sounds associated with them. They measure 
students’ ability to manipulate individual phonemes (sound units) within words, when, for 
example, students are asked to blend a list of phonemes into a word, segment a word into its 
corresponding phonemes, or identify the sounds which begin or end a word (Ritchey & Speece, 
2006). The relationships between these constructs in English are well-documented in the 
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research literature. In early readers, ability to identify letter names and the sounds that letters 
make predicts phonemic awareness. Phonemic awareness predicts fluency, and low fluency is a 
strong predictor of difficulties in reading (National Reading Panel, 2000). 

As student reading skill progresses, it is necessary to use different reading measures to be 
able to continue to track the progress students are making as developing readers. Oral reading 
fluency, which measures a combination of students’ sight vocabulary and their ability to decode 
novel words rapidly and accurately, is consistently identified in the literature as one of the best 
predictors of student reading comprehension in the early grades (Graves, Plasencia-Peinado, 
Deno, & Johnson, 2005; Hasbrouck & Tindal, 2005). Eventually, however, the information 
provided by measures of oral reading fluency is limited. Readers attain a fluency threshold that 
enables them to attend to comprehension rather than decoding (Ehri, 1991, 2005). Once this 
threshold has been reached, fluency is no longer sensitive to increases in reading comprehension. 
At this point, one must turn to measures designed to assess comprehension more directly. 
Although this technical report provides information specifically related to the Word and Passage 
Reading Fluency measures developed for use in our Progress Monitoring assessment system, it is 
important to provide an overview of the complete system so readers can understand how the 
fluency measures fit into the system as a whole. 

The Measures that Comprise Our Complete Assessment System 

Based on previous empirical studies of early literacy assessment (see, for example, the 
report from the National Reading Panel, 2000), we decided to develop two measures of 
alphabetic principle (Letter Names and Letter Sounds), one measure of Phonological Awareness 
(Phoneme Segmenting), two measures of fluency (Word Reading Fluency and Passage Reading 
Fluency), and one measure of comprehension (Multiple Choice Reading Comprehension). The 
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specific technical specifications for the Word and Passage Reading Fluency measure are 
described in the methods section of this technical report. First, we describe the specific 
requirements related to the intended use of the measures in our assessment system. 

When one is interested in monitoring the progress students are making in attaining 
specific skills, it is important to have sufficient measures to sample student performance 
frequently. Thus, our goal was to create 20 alternate forms of each measure in our assessment 
system at each grade level where the measure was designed to be used (see Table 1). Because 
these alternate forms are designed to be used for progress monitoring, it is essential that all forms 
of a particular measure in a given grade level be both sensitive to showing growth in a discrete 
skill area over short periods of time (1-2 weeks of instruction) and comparable in difficulty. 
These two equally important needs informed all parts of our measurement development effort: 
the construction of the technical specifications for each of the measures, the design of the studies 
used to gather data on item and test functioning, the analytic approaches we used to interpret the 
results of the pilot studies, and subsequent revision of the measures. In all cases, we sought 
approaches that would provide us with enough information to evaluate the sensitivity of the 
individual measures to detect small differences in student performance and the comparability of 
the different forms of each measure to allow for meaningful interpretation of growth over time. 
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Table 1 



Distribution of the Measures Across the Grades 



Measure 


Grade 


Letter 

Names 


Letter 

Sounds 


Phoneme 

Segmenting 


Word 

Reading 

Lluency 


Passage 

Reading 

Lluency 


MC 

Reading 

Comp 


Kindergarten 


X* 


X 


X 


X 






Grade 1 


X 


X 


X 


X 


X 




Grade 2 






X 


X 


X 




Grade 3 








X 


X 


X 


Grade 4 










X 


X 


Grade 5 

_ 










X 


X 



Note: Each “X” represents 20 alternate forms of the measure for that grade level. 



In the section that follows, we describe the piloting methods used to gather information on the 
relative difficulty of different forms of the fifth-grade passage reading fluency measures. 

The Passage Reading Fluency Measure 

The Passage Reading measure tests students’ ability to read connected narrative text 
accurately. In this individually-administered measure, students are shown a short narrative 
passage (approximately 250 words) printed on one side of a single sheet of paper and given 60 
seconds to read as much of the passage as they can. A trained assessor follows along as the 
student reads, indicating on his/her own test protocol each word the student reads incorrectly and 
prompting the student to go on if he/she hesitates for more than three seconds. Student self- 
corrections are counted as correct responses. At the end of the allotted time, the assessor marks 
the last word read and calculates the total number of words read correctly to arrive at the 
student’s score, words read correctly in one minute. 
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Methods 

Our goal was to create 20 alternate forms of each measure at each grade level where the 
measure was designed to be used (see Table 1). Because these alternate forms will be used for 
progress monitoring, it is essential that all forms of a particular measure in a given grade level be 
comparable in difficulty. The methods we used to create these comparable alternate forms are 
described in the following section. 

Creating Alternate Forms of the Passage Reading Fluency Measures 

The passages used in the Passage Reading Fluency measures were all written specifically 
for use in this progress monitoring assessment system. All 20 passages were written by graduate 
students enrolled in College of Education courses in the spring and summer of 2007. Passage 
writers followed written test specifications (see Appendix A). All passages underwent a four- 
stage review process. First, the lead author, who holds a Bachelor’s of Arts degree in English and 
is a National Board for Professional Teaching Standards certified English teacher, reviewed each 
passage. She edited the passages for grammatical correctness and grade-level appropriateness. 
Then, two graduate students edited for formatting consistency. They divided each passage into 
three paragraphs of approximately even length and checked the readability of each paragraph 
using the Flesch-Kinkaid readability index feature available on Microsoft Word. Each fifth-grade 
paragraph was adjusted as needed to create three paragraphs with a readability level between 5.4 
and 5.6. 

Third, each passage was reviewed by a teacher with a minimum of three years’ teaching 
experience at that particular grade level to ensure the topics, wording, and style were appropriate 
for the target grade levels. Finally, passages were sent back to the lead author for a final review 
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to ensure that they still met test specifications. Once the review process was complete, the 
passages were printed on 8 Vi by 1 1 inch paper for use during the pilot testing process. 

Analysis of the Passage Reading Fluency Measures 

Unlike the other measures in the Progress Monitoring Assessment System, the Passage 
Reading measures are not appropriate for analysis using Item Response Theory, as each form of 
the measure would be considered a single ‘item’ with a score ranging from 0 - 255. Thus, we 
analyzed the Passage Reading Fluency forms using classical statistics. More specifically, we 
analyzed both correlations and mean differences between the different forms of the measures 
using a repeated measures analysis. To increase the reliability of our score interpretations, we 
administered all 20 alternate forms of each grade level passage to the same group of students, 
over the course of one week (each student completed five Passage Reading Fluency forms per 
day). 

Results 

The fifth-grade Passage Reading Fluency measures were pilot tested in February of 2008. 
We present the results of this pilot testing below. 

Results of the Passage Reading Fluency Pilot Testing 

Descriptive statistics from the fifth-grade Passage Reading Fluency pilot study are 
presented in Table 2. Correlations between each of the 20 forms are presented in Table 3. 
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Table 2 



Descriptive Statistics for Grade 5 Passage Reading Measures 



Passage 


Topic 


n 


M 


SD 


Gr5PRF_l 


Crossword puzzles 


74 


134.96 


32.239 


Gr5PRF_2 


Playing baseball 


74 


156.00 


38.253 


Gr5PRF_3 


New baby brother 


74 


164.15 


42.690 


Gr5PRF_4 


Moving to a new country 


74 


164.53 


36.961 


Gr5PRF_5 


Zoo animals 


74 


162.54 


39.541 


Gr5PRF_6 


Seating charts 


74 


161.82 


39.808 


Gr5PRF_7 


Astronaut 


74 


165.95 


34.937 


Gr5PRF_8 


Jen’s crush 


74 


154.07 


40.710 


Gr5PRF_9 


School pranks 


74 


168.99 


38.482 


Gr5PRF_10 


Piano recital 


74 


165.89 


43.019 


Gr5PRF_ll 


Spider snack 


70 


160.26 


41.254 


Gr5PRF_12 


Puppy of her own 


70 


165.74 


42.009 


Gr5PRF_13 


Time machine 


70 


153.63 


41.202 


Gr5PRF_14 


Real life hero 


70 


149.86 


36.328 


Gr5PRF_15 


Spelling quiz 


70 


160.31 


33.414 


Gr5PRF_16 


Tooth aches 


68 


164.57 


38.255 


Gr5PRF_17 


Talent show 


69 


163.43 


37.618 


Gr5PRF_18 


Book club 


69 


162.86 


38.926 


Gr5PRF_19 


New bike 


68 


160.31 


37.912 


Gr5PRF_20 


Career choices 


69 


170.86 


38.611 
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Correlations between Each of the Fifth-Grade Passage Reading Fluency Forms 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


1 




















2 


.89** 


1 


















3 


89** 


94** 


1 
















4 


.86** 


92** 


93** 


1 














5 


.88** 


93** 


94** 


93** 


1 












6 


92** 


94** 


94** 


93** 


92** 


1 










7 


89** 


92** 


92** 


89** 


91** 


93** 


1 








8 


90** 


91** 


92** 


91** 


90** 


94** 


91** 


1 






9 


89** 


94** 


93** 


92** 


93** 


.95** 


92** 


.96** 


1 




10 


.86** 


94** 


92** 


92** 


91** 


94** 


93** 


93** 


94** 


1 


11 


§7** 


93** 


94** 


93** 


93** 


94** 


92** 


93** 


92** 


94** 


12 


.84** 


90** 


92** 


89** 


90** 


94** 


92** 


92** 


92** 


93** 


13 


.82** 


89** 


91** 


.88** 


89** 


93** 


.85** 


.86** 


.86** 


.88** 


14 


.80** 


.83** 


.85** 


.81** 


.84** 


37** 


00 

* 

* 


.86** 


.85** 


37** 


15 


.80** 


.85** 


.85** 


.83** 


.84** 


.85** 


.83** 


.84** 


84** 


.86** 


16 


.83** 


89** 


g7** 


.85** 


37** 


.85** 


91** 


.95** 


.95** 


91** 


17 


§7** 


93** 


92** 


91** 


.89** 


.95** 


37** 


37** 


.88** 


92** 


18 


.83** 


.85** 


.86** 


.83** 


.82** 


37** 


37** 


37** 


37** 


.88** 


19 


.85** 


91** 


92** 


89** 


92** 


92** 


89** 


90** 


93** 


93** 


20 


.80** 


89** 


.88** 


.86** 


.89** 


89** 


37** 


.88** 


92** 


90** 



■ 



Fluency 



ii 


12 


13 


14 


15 


16 


17 


18 


19 












































































































































































































i 




















94** 


1 


















.85** 


37** 


1 
















89** 


.88** 


.81** 


1 














.85** 


.81** 


.76** 


79** 


1 












93** 


91** 


89** 


.88** 


OO 

* 

* 


1 










.85** 


.85** 


.81** 


79** 


73** 


.88** 


1 








93** 


.85** 


.85** 


.81** 


79** 


73** 


.88** 


1 






.95** 


94** 


90** 


37** 


37** 


90** 


93** 


.83** 


1 




89** 


89** 


37** 


.83** 


.81** 


.81** 


.88** 


.85** 


90** 


i 
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Discussion 

Creating Alternate Forms of the Passage Reading Fluency Measures 

We used the information about passage difficulty to modify the passages and bring them 
into closer alignment. First, we identified the nine passages that were most similar in difficulty. 
Then, we increased the difficulty of the passages that were too easy (based on more average 
words read correctly than the other passages included in the pilot testing) and decreased the 
difficulty of the passages that were too challenging (based on fewer average words read correctly 
than the other passages included in the pilot testing). The changes made to the fifth-grade 
passages are indicated in Table 4. 

Table 4 



Changes Made to Grade 5 Passage Reading Measures 



Passage 


Changes made to the passage 




Made less challenging by changing Mr. Jacobson to Mr. Jacobs, ‘assigned to 


Gr5PRF_l 


groups’ to ‘put in groups’, replacing ‘vertically and horizontally’ with ‘up and 
down and across the paper’, and changing several ‘the students’ to ‘they’. 


GrSPR F 9 


Made slightly less challenging by changing ‘bannister’ to ‘railing’, ‘dreamt’ to 




‘had dreamed’, and ‘sluggishly’ to ‘slowly’ 


Gr5PRF_3 


No changes made 


Gr5PRF_4 


No changes made 


Gr5PRF_5 


No changes made 


Gr5PRF_6 


No changes made 


Gr5PRF_7 


No changes made 


Gr5PRF_8 


Made slightly easier by changing ‘Jasmine’ to ‘Jen’ 


Gr5PRF 9 


Made slightly more difficult by changing Jose to Mario, lengthening a few 




sentences. 
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Table 4 

Changes Made to Grade 5 Passage Reading Measures ( Continued ) 



Passage 



Changes made to the passage 



Gr5PRF_10 No changes made 



Gr5PRF_ll No changes made 



Gr5PRF_12 No changes made 



Made slightly easier by changing ‘garage’ to ‘house’, ‘bin’ to ‘box’, ‘Derrick’ 
to ‘Danny’, and ‘came across’ to ‘found’ 



Gr5PRF 13 



Made easier by changing ‘dreamt’ to ‘dreamed’, ‘characters’ to ‘people’, 
‘daydream’ to ‘dream’, ‘dreaming’ to ‘thinking’, ‘Lancaster’ to ‘Lee’ 



Gr5PRF 14 



Gr5PRF_15 No changes made 



Gr5PRF_16 No changes made 



Gr5PRF_17 No changes made 



Gr5PRF_18 No changes made 



Gr5PRF_19 No changes made 



Made more challenging by replacing ‘Jason’ with ‘Jeremy’, ‘all of his friends 
knew’ with ‘all of his friends already had career plans’, ‘Brian’ with ‘Amanda’, 
and ‘Jamie’ with ‘Jimmy’ 



Gr5PRF 20 



This revision process resulted in 20 comparable forms. For the Student Form of the measures, we 
used size 14 Verdana font (see Appendix B). The Assessor Copy of each of the forms includes 
administration and scoring directions as well as a smaller version of the student measure (see 
Appendix C). All forms of the measures were then loaded to the EasyCBM website for web- 



based access. 
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Appendix A 

Test Specifications for Creating Passage Reading Fluency Measures 

Instrument Development: ORF Passages 

Goal: To develop a series of reading assessments that can be used for progress monitoring and 
tracking RTI data for 1st- through 4 th - grade students at risk for reading failure. 

Overall Task : Write 25 ORF passages (each 250 words long) for 1 st , 2 nd , 3 rd , and 4 th -grade 
students (for a total of 100 ORF passages). 

Grade 1 ORF passages should be written at a 1.5 grade level. 

Grade 2 ORF passages should be written at a 2.5 grade level. 

Grade 3 ORF passages should be written at a 3.5 grade level. 

Grade 4 ORF passages should be written at a 4.5 grade level. 

Grade 4 ORF Passages should be written at a 5.5 grade level. 

• Each ORF passage should ‘stand alone’ with no reference to any other ORF passage. 

• Each ORF passage should tell a story. 

• ORF passages should contain no dialogue. 

• Give each ORF passage an appropriate title. 

• Include Word Count, on each ORF passage. 

• Email each ORF passage to me AS SOON AS YOU COMPFETE IT so I can keep track of 
our overall numbers. 



I’ve provided an example ORF passage from each of the grade levels we’re writing them for on 
the next few pages of this document. Please fa mi liarize yourself with the basic format / 
approach, and then jump right into writing! 

Thank you ! 



-Julie 
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Appendix B 

Example Fifth Grade Passage Reading Fluency Test: Student Copy 

Jeremy had no idea what he wanted to be when he 
grew up. It seemed like all of his friends already had career 
plans. His friend Amanda wanted to be a doctor, and his friend 
Jimmy wanted to be a police officer. His sister and his brother 
wanted to be teachers. But none of these jobs seemed right to 
Jeremy. His mother told him to try to think of a job that was 
related to his favorite subject in school. But Jeremy didn't think 
anything he had studied was very exciting. He knew that he had 
time to decide what kind of job he wanted, but he still wished 
that he had an idea like everyone else. 

One day, Jeremy's class went on a field trip to the science 
museum. There was a special exhibit there called The Human 
Brain. Jeremy was excited because he had never learned about 
the brain before. As soon as Jeremy walked into the museum, he 
was amazed. There were models and pictures of brains. There 
was a woman talking about how different parts of the brain have 
different roles. At another station, a man was talking about how 
differences in brains are what make people unique. 

After his day at the museum, Jeremy knew what he wanted 
to do when he grew up. He wanted to do research on the brain. 
Maybe someday he could even work at a museum and teach 
students everything he had learned. Jeremy couldn't wait to go 
home and tell his family about his dream. 
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Appendix C 

Example Fifth-Grade Passage Reading Fluency Test: Assessor Copy 

1 . Place the first passage without numbers in front of the student. Point to any names in the document and tell 
the student how to say the name. Then say: 

“I want you to read this story to me. You’ll have 1 minute to read as much as you can. When I say “begin,” start 
reading aloud at the top of the page. Do your best reading. If you have trouble with a word, I’ll tell it to you. Do you 
have any questions? Begin.” 

2. Start the timer. 

3. While the student is reading, mark errors with a slash (/). 

4. At 1 minute, say “stop.” 

5. Mark the last word read with a bracket (]). 

Jeremy had no idea what he wanted to be when he grew up. It 14 

seemed like all of his friends already had career plans. His friend 26 

Amanda wanted to be a doctor, and his friend J immy wanted to be a 40 

police officer. His sister and his brother wanted to be teachers. But 52 

none of these jobs seemed right to J eremy. His mother told him to try 66 

to think of a job that was related to his favorite subject in school. But 81 

Jeremy didn't think anything he had studied was very exciting. He 92 

knew that he had time to decide what kind of job he wanted, but he 107 

still wished that he had an idea like everyone else. 117 

One day, Jeremy's class went on a field trip to the science 129 

museum. There was a special exhibit there called The Human Brain. 140 

Jeremy was excited because he had never learned about the brain 151 

before. As soon as Jeremy walked into the museum, he was amazed. 163 

There were models and pictures of brains. There was a woman talking 175 
about how different parts of the brain have different roles. At another 187 

station, a man was talking about how differences in brains are what 199 

make people unique. 202 

After his day at the museum, Jeremy knew what he wanted to do 215 

when he grew up. He wanted to do research on the brain. Maybe 228 

someday he could even work at a museum and teach students 239 

everything he had learned. J eremy couldn't wait to go home and tell 251 

his family about his dream. 256 

Total Words Read: # of Errors: 
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